The best way around the CBD is by taxi as the city is fairly small and taxi fare is quite cheap. Taxis are highly recommended against if crossing the bridge as Yeosu has some of the worst traffic jams in Korea at this bottleneck, despite its tiny size. Even by bus, it could be optimal to get off, walk the bridge and hop back on the previous bus at the other end. Ridiculous.
By bus
The bus system is very convenient, but a little difficult to utilize. Each bus stop should have a posting of the buses that will arrive and their respective stops but most have been inexplicably vandalised. Those that do still exist are strictly in Korean.
Once south of the main bridge, most buses actually take a similar route so it can be easy to hop on and off in whichever direction to wish to head. The one exception to this is a branch where half the buses turn south-east to stop at the hermitage whereas the rest continue south around the island chain.
The "magic number" bus, if you can call it that, is the 111. This bus travels a zig-zagging route from the Intercity bus Terminal to Hyangiram Hermitage via Jinnamgwan and the harbour, crossing the Dolsan Bridge and passing many of the tourism attractions on the southern islands.